REPORT  DOCUMENTATION  PAGE 


Form  Approved  OMB  No.  0704-0188 


Public  reporting  burden  for  this  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions  searching  exis ting  data  s°urces 
gathering  and  maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  o^er  aspect  of  this 
collection  of  information,  including  suggestions  for  reducing  this  burden  to  Washington  Headquarters  Services  ^,r®ctorate  for  J-Jf 1  ^  Je 

Davis  Highway  Suite  1204  Arlington,  VA  22202-4302,  and  to  the  Office  of  Management  and  Budget,  Paperwork  Reduction  Project  (0704-0188)1Washington,  DC  20503. - 

1.  AGENCY  USE  ONLY  (Leave  blank)  I  2.  REPORT  DATE  I  3.  REPORT  TYPE  AND  DATES  COVERED 


24-April-2000 


4.  TITLE  AND  SUBTITLE 


Final  Report 


5.  FUNDING  NUMBERS 


Super-fast  Intelligent  Target  Evaluation  (SITE) 


F61775-99-WE083 


6.  AUTHOR(S) 


Dr.  Tamas  Roska 


7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

SzTAKI 
Kende-u.  13 
Budapest  H  1111 
Hungary 


8.  PERFORMING  ORGANIZATION 
REPORT  NUMBER 


9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 
EOARD 

PSC  802  BOX  14 
FPO  09499-0200 


10.  SPONSORING/MONITORING 
AGENCY  REPORT  NUMBER 

SPC  99-4083 


11.  SUPPLEMENTARY  NOTES 


12a.  DISTRIBUTION/AVAILABILITY  STATEMENT 

Approved  for  public  release;  distribution  is  unlimited. 


12b.  DISTRIBUTION  CODE 


13.  ABSTRACT  (Maximum  200  words) 

This  report  results  from  a  contract  tasking  SzTAKI  as  follows:  Contractor  will  conduct  a  feasibility  study  on  the  system  idea  for  the  SITE  unit  to 
detect  targets  with  predetermined  morphology  under  extremely  difficult  visibility  conditions.  Study  will  involve  the  development  of  an  algorithm 
and  an  optical  lensing  system  to  test  the  concept. 


14.  SUBJECT  TERMS 


EOARD,  Neural  Networks 


15.  NUMBER  OF  PAGES 

_ 23 _ 

16.  PRICE  CODE 


17.  SECURITY  CLASSIFICATION 
OF  REPORT 

UNCLASSIFIED 
NSN  7540-01-280-5500 


18.  SECURITY  CLASSIFICATION  I  19,  SECURITY  CLASSIFICATION 
OF  THIS  PAGE  OF  ABSTRACT 


UNCLASSIFIED 


UNCLASSIFIED 


20.  LIMITATION  OF  ABSTRACT 


Standard  Form  298  (Rev.  2-89) 
Prescribed  by  ANSI  Std.  239-18 
298-102 


20000913  063 


EOARD 


Super-fast  Intelligent  Target  Evaluation 

(SITE) 

FINAL  REPORT 

Feasibility  study  and  R&D  program 


SPC  99-4083 

Contract  order  number:  F  61775-99-WE083 
Dept.  Air  Force,  European  Office  of  Aerospace  Research  and  Development  (EOARD) 

Tamas  Roska,  Ph.D.,  D.Sc,  Prof.,  Principal  Investigator 

Analogical  and  Neural  Computing  Laboratory  (ANCL), 

Computer  and  Automation  Institute  of  the  Flungarian  Academy  of  Sciences 

(MTA-SZTAKI) 

Kende-u.  13-17,  Budapest,  H-l  1 1 1,  HUNGARY 
Phone:  +36  1  209  5263 Fax:  +36  1  209  5264 
e-mail:  roska@sztaki.hu 


March  2000,  Budapest 


flQ  F 


Table  of  Contents 


1.  Executive  Summary _ 3 

2.  Introduction _ 4 

3.  Target  Specification _ 5 

4.  Proposed  Architecture _ 6 

5.  Required  Technologies  and  Their  Approximate  Specifications _ 10 

6.  Work  Plan,  Phases  and  Contributing  Parties _ 14 

7.  References _ 16 


Appendix 


2 


1.  Executive  Summary 


Recently,  dramatic  failures  in  military  and  industry  have  proved  the  vulnerability  of 

present  day  sensor-computer  technologies.  In  spite  of  witnessing  the  sensor-revolution  in 

these  days,  the  integration  of  multimodal  sensor  arrays  and  computers  are  in  their  infancy. 

In  this  project,  following  successful  theoretical  and  experimental  work  on  a  new 

computer  paradigm,  called  analogic  cellular  (CNN)  computers  and  related  technologies,  a 

major  effort  is  planned  to  achieve  a  new  level  of  quality  and  computing  power  in  the  sensor 

computer  arena.  By  doing  this,  a  major  bottleneck,  built  into  any  existing  high  speed  target 

recognition  and  tracking  system,  is  removed.  This  bottleneck  is  the  separation  of  sensing  and 

computing  due  to  the  analog  sensing  and  digital  processing.  In  our  new  sensor-computer 

framework  a  genuine  dynamic  interaction  of  the  two  processes  is  integrated,  like  in  the  most 

successful  living  creatures.  Analogic  means  :combining  analog  array  dynamics  and  logic. 

Analogic  cellular  computers  has  became  recently  not  only  of  theoretical  interest  and  a 

neuromorphic  modeling  tool.  The  measured  TeraOPS  computing  power,  in  a  focal  plane  CNN 

Universal  Machine  chip  with  4096  processors  using  a  conservative  0.5  micron  standard 
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CMOS  technology,  on  1  cm  at  1W,  manifested  a  beginning  of  a  new  era  in  sensory  array 
computers.  In  addition,  a  target  detection  experiment  with  a  breathtaking  10,  000  frames  per 
second  has  been  achieved  during  the  last  few  weeks  using  these  chips  in  a  test  system. 
Meanwhile,  breakthroughs  have  been  witnessed  in  the  following  connecting  areas: 

•  Deep  submicron  mixed  mode  CMOS  design  with  visible  wavelength  sensors 

•  MEMS  type  long  wave  IR  sensor  arrays 

•  Vertical  packaging  of  VLSI  chips  and  various  new  lens  technologies 

•  Spatio-temporal  analogic  computing  with  adaptation  and  learning 

•  Efficient  computational  infrastructure  for  analogic  hardware/software  systems  integration 
The  goal  of  the  present  project  is  to  make  a  Super-fast  Intelligent  Target  Evaluation 

(SITE)  unit  by  exploiting  the  synergy  of  all  the  above  mentioned  technologies,  leading  to  a 
walkman-size,  programmably-adaptive,  multisensor  supercomputer.  This  unit  is  differing 
dramatically  from  any  presently  known  technologies  and  offering  about  3  orders  of  magnitude 
speed/size/  or  power  consumption  advantages  providing  a  new  generation  of  cheap  mission 
critical  devices.  These  devices,  with  about  50  to  100  TeraOps  speed,  will  achieve  context  and 
content  dependent,  dynamic,  target  detecting  and  tracking,  leading  to  a  new  generation  of 
equipments  and  methodologies. 
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The  R&D  program  of  the  next  Phase  II  has  two  parts.  Phase  IIA  will  implement  the  new 
concept  by  integrating  the  upgraded  existing  technologies  and  pursue  research  to  explore 
more  advanced  properties,  synergies,  and  application  areas,  including  airborne  and  other 
mission  critical  testing.  Phase  IIB  will  integrate  the  advanced  research  results  of  Phase  IIA  in 
a  vertically  integrated  system. 

2.  Introduction 

Since  a  few  years,  a  new  computing  technology  has  been  emerging,  drastically  different 
from  any  previous  one.  Its  elementary  instructions  are  analog  spatio-temporal  waves 
combined  with  logic,  the  processor  array  is  fully  parallel,  hence,  it  is  also  called  analogic 
cellular  (CNN)  computer.  Since  the  elementary  instructions  are  the  ones  most  difficult  to 
implement  on  any  digital  computer,  solving  a  nonlinear  PDE  on  a  finite  grid,  doubts  have 
been  always  present  about  the  feasibility  of  physical  implementation  of  this  revolutionary 
technology. 

In  December  2000,  in  Washington  DC,  during  a  Technology  Assessment  Meeting 
organized  by  ONR  and  other  agencies,  the  proof  of  concept  was  put  on  the  table:  complex 
tasks  were  solved  by  an  analogic  4096  processor  chip  8000  time  faster  than  by  a  Pentium  II 
machine  with  a  fraction  of  size  and  power  dissipation.  Moreover,  very  high  speed  target 
detection  can  now  be  demonstrated  with  10,  000  frame  per  second. 

The  original  ideas  had  been  invented  in  Berkeley  [1-4]  and  the  new  chips  and  computing 
infrastructure  have  been  developed  in  Seville  [7-9]  and  Budapest  [5,  16],  in  a  collaborative 
effort  of  the  three  groups. 

The  new  computing  paradigm  is  especially  efficient  in  sensor  computers  [5]  where  on- 
chip  sensors  are  dynamically  interacting  with  on  chip  analogic  array  computers.  The  first 
experimental  focal  plane  analogic  computers  show  unprecedented  computing  power 
(TeraOPS)  and  algorithmic  flexibility. 

The  aim  of  the  present  project  to  make  research  and  experimental  efforts  to  test  the 
ultimate  limits  of  present  day  and  future  technologies  in  achieving  Superfast  Intelligent  Target 
Evaluation  (SITE).  The  main  areas  of  R&D  are  as  follows. 

•  Deep  submicron  mixed  mode  analogic  cellular  array  computer  chips  with  on  chip  adaptive 
sensors. 

•  Long  wave  IR  sensors  to  be  mounted  on  the  analogic  cellular  computer  CMOS  chips. 

•  Vertical  and  other  packaging  technologies  to  make  the  sensor  supercomputers  smaller 
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•  Physical  integration,  including  various  lens  technologies  to  explore  additional  capabilities 

•  Analogic  and  standard  Software  R&D  and  integration  for  solving  complex  target  detection 
and  tracking  problems 

•  Functional  system  integration  techniques  to  make  airborne  and  on  site  units  and  to  test  them 
in  mission  critical  real  life  situations 

This  is  a  typical  dual-  or  multi-use  technology.  Besides  the  mission  critical  applications, 
medical,  industrial,  security  and  many  other  fields  would  gain  a  lot. 

3.  Target  Specification 

The  Super-fast  Intelligent  Target  Evaluation  (SITE)  system  represents  the  presently 
available  ultimate  speed-power-area  performance  of  multi-modal  sensor  supercomputers  in  a 
camera  size  unit. 

The  estimated  maximum  computing  power  is  about  50-100  TeraOPS. 

The  maximum  frame  rate  for  target  detection  is  about  50,  000  frames  per  second. 

The  maximum  pixel  resolution  is  about  0.25  million  pixels  depending  on  the  frame  rate. 

Most  of  the  building  blocks  of  the  system  have  been  tested  in  details,  although  not  at  the 
targeted  complexity  and  scale,  and  not  in  this  system  configuration. 

The  architecture  is  based  on  an  advanced  version  of  a  new  Analogic  Cellular  (CNN) 
Computer  paradigm  and  relies  on  the  CNN  Universal  Machine  (CNN-UM)  architecture.  Its 
extended,  adaptive  cells  are  used  to  integrate  multi-modal  sensory  and  computing  arrays. 
Dynamic  intelligence  is  achieved  in  an  unparalleled  way  and  technology. 

To  make  this  system  a  reality,  that  is  a  prototype,  a  focused  and  organized  R&D  is  needed 
in  all  the  relevant  areas: 

1.  Deep  submicron  (-0.18  micron)  mixed  mode  CMOS  design  and  processing 

2.  Microsensor  and  MEMS  arrays  for  multispectral  light  (visible,  IR,  LWIR,  etc.) 

3.  Vertical  packaging  with  internal  vias  and/or  MCM  technology 

4.  System  integration  on  the  physical  level,  including  various  lens  technologies 

5.  Analogic  software  R&D  for  sensor-computers  and  software  integration 

6.  System  integration  on  the  functional  level  and  mission  critical  testing 

The  synergy  of  these  areas  would  lead  to  a  new  quality  and  a  new  class  of  technical 
specifications  in  general,  making  possible  to  directly  integrate  sensing  and  computing  as  well 
as  spatio-temporal  flow-control  for  extremely  complex  target  tracking  problems. 
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4.  Proposed  Architecture 


The  input  module  of  a  SITE  system  contains  a  Multi-modal  Microsensor  Array  (MMA). 
This  is  an  array  of  cells,  each  of  which  contains  several  sensor  elements  for  detecting  different 
sensory  modalities.  The  simplest  one  is  a  color  camera.  “Color”  may  mean  various  sensors 
and  filters  for  different  spectra,  polarization,  sensitivity,  and  speed.  The  local  sensitivity  of 
each  sensor  could  be  controlled  depending  on  the  local  features  of  pre-processed  images. 
Hence,  sensing  and  processing/computing  is  integrated  in  the  input  module. 

A  cell  in  the  input  module  is  shown  in  Fig.  1.  The  local  intensity  of  the  sensor  is 
controlled  either  by  a  signal  coming  from  the  sensor  cell  or  from  the  subsequent  layer:  a 
simple  Analogic  Cellular  (CNN)  Computing  chip  (ACC). 

The  extended  cell  of  the  input  module  consists  of  a  very  simple  CNN  cell,  some  local 
analog  memory  (LAM),  local  logic  memory  (LLM),  as  well  as  a  local  switching  unit  (LSU), 
and  a  local  analogic  control  unit  (LACU).  The  sensitivity/gain  of  each  sensory  cell  can  be 
controlled  by  a  control  signal  y,  \2,  ...vn  and  the  cell  ouputs  y ,  yi,  ...y^  are  either  stored 
locally  in  LAM  or  LLM  or  they  are  sent  to  the  subsequent  layer.  The  signals  y,  y>,  . .  .vn  are 
coming  either  from  the  LACU  or  from  the  subsequent  layer.  This  extended  cell  implements 
just  a  few  CNN  templates:  the  goal  is  to  adjust  the  sensors  and  possibly  to  detect  the  region  of 
interest  (ROI)  . 


Figure  1  A  schematic  of  an  input  module  cell  with  3  sensors.  S|-S3  are  the  sensors  with 
sensitivity/gain  control  through  signals  V1-V3  and  output  signals  yi-y3. 

Following  the  input  module,  the  next  few  layers/chips  perform  the  more  sophisticated 
processing  tasks.  Two  main  versions  of  SITE  units  are  to  be  developed.  Sketches  of  one 
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specific  version  of  each  phases,  SITE-1  and  SITE-2,  with  the  main  components  are  shown  in 
Fig.  2  and  3,  respectively. 

SITE-1  is  a  single  module  version  without  or  with  modest  3D  packaging. 

SITE-2  is  a  multi-chip  unit  with  3  D  packaging  and  or  MCM  technology. 

In  both  cases,  the  sensory  array  is  integrated  in  different  ways,  using  different  vertical 
integration  technologies. 


area*— 

^  acc-a| 

> 

fj 

/  :  //? 

r—  ACC-B 

MMA: 

Multimode 
Microsensor 
Array 

}  Pre¬ 
processing 

with  event 

... 

recognition 


Main 
processing 
unit  /object 
recognition 
and 

tracking  / 


Connector:  interface 
to  digital  processing 
and  control 


to  the  Analogic  CNN  Engine  Board 
ACC:  Analogic  Cellular  (CNN)  Computer  chip 


Figure  2  SITE-1:  a  single  module  version. 
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MMA 


Figure  3  SITE-2:  a  multi  chip  module  version. 

A  single  cell  processor  of  the  Analogic  Cellular  (CNN)  Computer  chip  (ACC)  main 
processing  unit  is  shown  in  Fig.  4.  This  may  be  a  complex  cell,  a  cell  of  a  multi  layer  CNN- 
UM  architecture  with  local  adaptation,  consisting  of  the  following  main  parts: 

•  Connections  from  input  module 

•  Extended  cells 

•  Template  adaptation  via  TCM 

•  Template  plasticity  via  potentiation  circuits  and  TCS 

Template  adaptation  is  generally  used  in  the  input  module.  The  sensor  parameters  of  the 
Multi-modal  Micro-sensor  Array  (MMA)  can  be  tuned,  adapted,  based  on  the  local  content  of 
the  scene,  including  the  range  of  the  sensed  signal.  Moreover,  sensors  can  be  tuned  to  be 
speed  sensitive,  depending  on  the  rate  of  change  of  the  signals. 

Template  plasticity  is  achieved  via  the  properties  of  a  sequence  of  frames  based  on 
some  plasticity  rules.  These  potentiation  rules  are  well  known  in  living  systems.  We  have 
identified  some  of  them  recently,  and  we  have  to  explore  them  for  our  artificial  systems. 
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Figure  4  An  adaptive  extended  cell  processor. 

The  main  parts  of  the  LAM  and  the  main  adaptation  mechanisms  in  the  adaptive 
extended  cell  are  as  follows. 


GACU  - 

Global  Analogic  Control  Unit 

LCCU  - 

Local  Control  and  Communication  Unit 

LAM 

Local  Analog  Memory 

LLM 

Local  Logical  Memory 

LLU 

Local  Logical  Unit 

LAOU  - 

Local  Analog  Output  Unit 

TCM 

Template  Control  Memory 

TCS 

Template  Control  Signal 

The  functional  diagram  of  the  SITE-1  unit  is  shown  in  Fig.  5.  Whether  using  sensorial 
layers,  complex  cells,  spatio-temporal  time  sharing  in  a  stored  program,  or  combining  them, 
depend  on  the  complexity  of  the  task. 
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Figure  5  The  functional  block  diagram  of  a  SITE-1  or  SITE-2  unit 

The  global  processing  unit  is  implemented  by  a  high-end  DSP  with  interface  circuits 
for  control  functions. 

5.  Required  Technologies  and  Their  Approximate  Specifications 

The  target  technical  specifications  for  the  different  technologies  are  as  follows: 


5.1  CMOS  chip  technology  with  0.25  and  0.18  micron,  double  supply  voltage  (analog  or 
high  speed  digital  and  low  power  logic),  single  poly,  and  about  8  metal  layers. 

In  this  part  of  the  project,  a  major  step  is  planned  in  mixed  mode  CMOS  design  going 
down  to  the  0.25  micron  technology  as  well  as  making  some  0.18  micron  experiments. 
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In  the  first  case  a  256x256  CNN-UM  chip  is  planned  with  extended  cells  to  make  some  local 
adaptation  and  control  to  the  sensors. 

In  the  second  case,  experimental  cells  with  Q18  micron  technology  will  be  designed  with 
focal  plane  adaptive  sensors.  Later,  a  complete  array  with  possible  300x400  cell  processors 
are  to  be  designed. 

Interfaces  to  the  LWIR  sensors  will  be  studied  and  developed. 

The  Seville  group  is  the  most  competitive  in  this  field  worldwide. 

5.2  Sensor  technology  on  silicon  including  normal  wavelength,  NIR,  LWIR,  f,  and 
possible  polarized  light,  with  possible  fixed  template  and/or  adaptive  CNN 
preprocessing.  This  is  a  new  direction  with  a  lot  of  new  challenges,  described  as  follows. 

The  most  important  advances  in  the  field  of  IR  uncooled  detector  have  been  achieved  on 
the  side  of  thermal  detectors  [16],  [17].  Micromachining  techniques  allowed  realizing  detector 
of  small  size  (50u)  and  less  with  low  thermal  capacity  and  large  thermal  insulation.  Several 
types  of  thermal  detectors  operating  at  room  temperature  have  been  realized  recently.  The 
focal  Plane  Array  (FPA)  developed  by  Honeywell  [18],  which  is,  uncooled  FPA  operating  in 
the  range  of  8-  12pm  range  is  basically  a  thermally  modulated  resistor. 

Depending  on  the  technology  used  for  integrating  the  detector  with  the  readout 
electronics,  FPA  can  be  either  hybrid  or  monolithic. 

Monolithic  arrays  for  example  the  Honeywell  is  obtained  by  directly  preparing  the 
detector  and  the  readout  circuit  on  the  same  substrate.  The  FPA  microbolometers  developed 
by  Honeywell  realize  large  array  (240x336)  sensors  operating  at  room  temperature.  Active 
element  is  made  of  Vanadium  Oxide.  The  FPA  readout  circuit  is  at  30HZ.  It  can  detect  temp. 
Difference  of  0.1k  between  two  elements  of  the  scene.  It  is  integrated  with  CMOS  circuits. 
The  main  disadvantage  of  using  Vanadium  Oxide  is  that  it  is  not  standard  material  in  IC 
technology,  and  it  requires  special  process  optimization  to  obtain  the  desired  activation 
energy,  resistivity  and  noise.  As  we  see,  it  is  very  possible  to  integrate  such  sensor  with  the 
CNN  chip.  The  main  problem  would  be  to  optimize  such  device  to  the  CNN  and  the 
bolometer  requirement. 

The  main  objectives  of  the  design  of  optimized  bolometer  are: 

1 .  Low  noise  equivalent  temperature  difference  lower  than  1  OOmk  at  a  frame  rate  of  30HZ 

2.  Material  must  be  compatible  with  standard  CMOS  process 
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3.  The  material  does  not  require  a  special  process  setup  as  in  the  case  of  Vanadium  Oxide. 

Moreover,  its  characteristics  can  be  easily  reproduced 

4.  The  complete  process  of  the  bolometer  must  be  performed  with  thermal  budget  implied  by 

standard  CMOS 

Recently  a  Polycrystalline  Silicon  Germanium  (Poly  Si  Ge)  alloy  was  proposed  as  a  new 
material  for  IR  bolometers  [19].  Any  bolometer  material  should  have  high  TCR  and  low  1/f 
noise.  The  request  for  low  thermal  conductance  also  influences  the  choice  of  the  material. 
Thermal  insulation  can  be  achieved  either  by  micromachining  suspended  membrane  or  by 
deposition  of  active  element  on  top  of  thermally  insulating  membrane. 

The  GWU  group  has  a  good  experience  in  designing  thermally  insulating  structure  that  is 
designing  membrane  suspended  on  top  of  CMOS  structure.  We  are  not  sure  about  the 
characteristics  needed  and  the  material  most  suitable  for  your  sensor  requirements.  An 
optimal  material  must  first  be  chosen.  If  we  will  consider  resistive  bolometers  Poly  Si  Ge  is  a 
possibility  as  also  Vanadium  Oxide.  This  requires  optimization  of  the  structure  and  the 
material  for  the  required  noise  level.  The  GWU  team  is  cooperating  with  other  groups  to 
decide  the  optimal  parameters  and  technological  variant.  As  to  the  visible  wavelength  sensors, 
3  types  of  CMOS  sensors  will  be  developed  and  interfaced  to  the  CNN  chips. 

5.3  Vertical  packaging  technology  including  flip  chip  and  multi-chip  modules  with 
internal  vias  on  about  a  50  micron  grid; 

Recent  advances  in  integrated  circuit  packaging  promise  a  new  level  of  integration  in 
analogic  CNN  processor  chips.  Multi  chip  module  technologies  and  vertical  integration 
techniques  are  now  hot  topics  when  the  roadmap  of  standard  and  cheap  CMOS  technologies 
approaches  physical  and  technological  limits. 

The  Urbana  Group  has  a  lot  of  experience  in  this  field  and  has  the  know  how  and  the 
cooperative  arrangements  to  make  a  major  advance  in  packaging  analogic  CNN  processor 
chips  to  achieve  a  much  higher  density. 

Unlike  the  Sugar  cube  technology  of  Irvine  Sensors  (a  BMDO  project),  here,  we  need  the 
vertical  connections  in  the  inner  part  of  the  chip,  with  about  50  micron  or  less  grid  size. 
Recent  advances  at  Raytheon-Boeing  as  well  as  at  Lucent  technologies  (with  optical  inner 
interconnects)  will  be  studied  as  promising  target  technologies. 
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5.4  System  integration  on  the  physical  level. 


In  addition  to  the  vertical  integration  of  VLSI  chips,  the  ultra  compact  visual 
supercomputer  chip  assemblies  need  a  new  physical  integration  technology  on  the  system 
level  as  well.  The  recently  emerging  industry  standard  PC104Plus  board  and  rack  technology 
is  planned.  This  means  that  a  few  floppy  disc  size  boards  will  be  mounted  together  in  a 
battery  or  supply  operated  environment.  The  final  result  will  be  a  walkman-size  unit  with 
supercomputer  processing  speed  and  100MHz  32  bit  communication  speed. 

In  some  cases,  when  the  fill  in  factor  is  to  be  increased,  micro-lens  technologies  are  to  be 
used.  Filling  in  the  optical  gaps  between  the  focal  plane  sensor-computer  chips  when  applying 
multi  chip  modules,  macro-lenses  are  needed.  The  recent  developments  in  panoramic  lens 
technologies  offer  high  speed  target  detection  without  any  rotating  units.  Hence,  for  special 
purpose  arrangements,  micro  lens  and  macro  lens  technologies  will  be  used  on  about  50 
micron  and  10-20  mm  grids,  respectively,  as  well  as  panoramic  lens  technology  will  be 
applied  for  fully  parallel  spherical  surveillance. 

These  optical  technologies  will  be  studied  in  the  context  of  speeding  up  as  well  as 
increasing  the  reliability  of  multisensor  target-  detection  and  -tracking  applications. 

5.5  Analogic  software  R&D  and  software  integration. 

The  architecture  of  the  adaptive,  complex  cell,  multi  sensor  and  multi  chip/board  system 
offer  a  new  ground  for  the  algorithm  and  software  design. 

•  Non-equilibrium  spatio-temporal  (NEST)  algorithms  [5]  will  be  the  norm,  rather  than  the 
special  case  in  this  new  environment.  Research  will  be  pursued  in  this  direction,  including 
active  wave  and  pattern  detection  systems  as  algorithmic  elements. 

•  The  computational  infrastructure  of  the  analogic  cellular  computers  should  be  extended  to 
include  the  adaptive  and  learning  processes  in  the  standard  software  environment  of  the 
compilers  and  interpreters. 

•  The  dynamic  interaction  of  analogic  cellular  and  standard  global  processing  will  be 
explored  and  implemented. 

The  Budapest  group  with  the  Berkeley  group  has  proved  to  be  a  leader  in  this  field,  and 
the  system  prototyping  is  planned  to  be  made  in  some  of  the  new  start-up  companies. 
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5.6  Functional  system  integration  and  mission  critical  testing 

At  the  end  of  both  phases  (Phase  IIA  and  Phase  IIB),  extensive  testing  is  to  be  made  in 
real  life,  airborne  and  ground  based,  environments. 

Studying  and  assessing  typical  application  classes,  one  or  two  functionally  integrated  units 
will  be  assembled. 

Independent  evaluation  by  other  groups  selected  by  the  contractor  agency  will  be  carried 
out  at  the  end  of  both  phases. 

6.  Work  Plan,  Phases  and  Contributing  Parties 

Work  Plan  and  Phases 

This  project  is  intended  to  be  completed  in  2  parts  and  6  tasks: 

Phase  II  A:  R&D  for  achieving  target  specifications  in  separate  chips  in  experimental  scale. 
Phase  II  B:  Scale-up  and  system  integration  on  the  physical,  software  and  functional  levels. 


Task  and 
Partners 

Phase  II  -  A 

Phase  II  -  B 

20 

Q3 

Ql 

20 

Q2 

01 

Q3 

Q4 

Ql 

20 

Q2 

Q4 

2003 

Ql 

1.  Mixed  mode 
chip  design: 
IMSE,  ANCL, 
MRL 

R 

Ci 

Ch 

O 

R 

Ci 

Ch 

IL 

0.25^1 

m 

0. 1 8|i 

2.  Sensors, 
CMOS:  IMSE, 

ANCL, 

MEMS:  MRL, 

IMSE,  ANCL 

R 

Ci 

Ch 

HI 

WM 

R 

Ci 

Ch 

R 

Ci 

H 

a 

R 

Ci 

Ch 

> 

E 

3.  Packaging: 
CSL,  ANCL, 

R 

Ci 

Ch 

a 

R 

Ci 

IMSE,  MRL 

■ 

4.  Physical 
Integration: 
ANCL.  CSL 

■ 

■ 

R 

Ci 

a 

Ci 

E3 

5.  Algorithm 
and  Software: 
ANCL,  NOEL 

R 

S 

a 

■ 

R 

wm 

Ml 

6.  Functional 
Integration 
and  testing: 
ANCL,  CSL 

R 

9 

R 

9 

m 

9 

In  the  quarters  indicated  by  the  columns,  the  fo 
completed: 

R  -  research  report,  Ci  -  circuit  design,  Ch  -  chip  or  board, 
S  -  software,  E  -  evaluation  report 


lowing  types  of  deliverables  will  be 
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Proposed  Budget  (in  1000  $): 


Phases  / 

Laboratories 

Phase  II  -  A 

Phase  II  -  B 

2000 

2001 

2002 

2003 

ANCL 

80+70  Eq 

210+50  Mf 

250+50  Mf 

80 

CSL 

1 10+50  Eq 

200+100  Mf 

250+  100  Mf 

IMSE 

150+80  Eq 

280+120  Mf 

280  +  130  Mf 

80 

MRL 

100+50  Eq 

200+100  Mf 

180  +  110  Mf 

80 

NOEL 

50 

100 

100 

Mf:  Manufacturing  cost 
Eq:  Equipment  cost 

As  a  summary  of  costs:  2780  +  250  Eq  +  760  Mfg  .  This  means  $3790k  total  cost 

Partners: 


ANCL:  Analogical  and  Neural  Computing  Systems  Laboratory  (ANCL)  and  the 

Neuromorphic  Information  Technology  Postgraduate  Center  (NIT),  Hungarian 
Academy  of  Sciences  and  the  Pazmany  University,  Budapest,  Hungary,  Prof. 
Tamas  Roska 

CSL:  Coordinated  Science  Laboratory  and  the  Department  of  Electrical  and  Computer 

Engineering,  University  of  Illinois,  Urbana-  Champaign,  USA,  Prof,  Steve  M. 
Kang 

IMSE:  Microelectronics  Laboratory,  Spanish  National  Research  Council  and  the 

University  of  Seville,  Seville,  Spain,  Prof.  AngeL  Rodriquez- Vazquez 
MRL:  MEMS  Research  Center,  Department  of  Electrical  Engineering,  George 

Washington  University,  USA  ,  Prof.  Mona  Zaghloul 
NOEL:  Nonlinear  Electronics  Laboratory,  Department  of  Electrical  Engineering  and 

Computer  Sciences,  University  of  California  at  Berkeley,  USA,  Prof.  Leon  O. 
Chua 
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Appendix  -  Examples  Showing  Qualitative  Actions  Using  Complex 
CNN  CELLS 


Some  detection  and  tracking  capabilities  are  illustrated  in  this  appendix.  The  problems 
are  solved  by  using  spatio-temporal  algorithms  running  on  CNN  Universal  Machine. 

Active  contour  detection  and  tracking 

Example  1  -  Detecting  the  contour  of  a  noisy  spiral-wave 

The  original  image  (left,  resolution:  64x64)  is  a  snapshot  of  a  rotating  spiral-wave.  This 
image  was  corrupted  by  additive  Gaussian  noise  and  the  contrast  degraded  to  create  the  CNN 
input  (right).  The  task  is  to  identify  the  contour  separating  the  light  and  dark  image  regions. 
The  problem  is  solved  by  a  trigger-wave  initiated  from  a  patch  in  the  light  region  (middle). 


Original  image  Initial  state  Noisy  input 


Consecutive  snapshots  from  the  CNN  output  -  simple  control: 


Consecutive  snapshots  from  the  CNN  output  -adaptive  control: 


r 


Example  2  -  Tracking  the  contour  of  the  left  ventricle 

This  fundamental  task  of  the  ultrasound  echocardiography  is  solved  on  a  "video-flow" 
of  the  echocardiography  machine.  25  typical  frames  (resolution:  64x64)  of  the  left  ventricle 
are  shown  bellow  -  supplied  as  the  input  of  the  CNN  Universal  chip. 
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The  processing  results  are  visualized  by  superimposing  the  extracted  contours  onto  the 
original  images.  Processing  a  frame  takes  about  250  psec  (measured  result  on  a  64x64  CNN 
Universal  Machine  chip).  The  predicted  time  performance  is  about  25  p  sec/frame  on  a 
128x128  CNN-UM  chip  fabricated  using  a  0.18-micron  technology. 
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Path  extraction  and  tracking 


Example  -  Finding  the  shortest  path 

A  spatio-temporal  algorithm  generating  trigger  waves  is  also  capable  of  finding  the 
shortest  path  in  labyrinth-like  landscapes  in  between  two  predefined  locations.  The  example 
bellow  illustrates  this  problem  on  a  128x128  input  containing  a  complex  path. 
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Target  tracking 

Example  -  Multiple-target  tracking  on  LWIR  images 

The  input  image  sequence  (20  frames  of  resolution  128x128)  is  taken  from  an  LWIR  “video¬ 
flow”.  The  video  shows  two  persons  hiding  and  moving  in  the  background.  The  task  is  to 
identify  and  track  these  “objects”. 


Measured  result  on  a  64x64  CNN  Universal  chip  (the  128x128  frames  were  processed  in 
blocks).  The  algorithm  combines  inter-frame  analysis  and  spatial  logic.  Red  -  accepted 
detection/tracking  result  on  the  actual  frame,  green  and  yellow  -  detection  results  extracted 
from  spatio-temporal  motion,  blue  -  prediction  based  on  previous  tracking  results. 


